Device and method for capturing, analyzing, and sending still and video images of the fundus during examination using an ophthalmoscope

ABSTRACT

The present invention is directed to a medical imaging device attachment with onboard sensor array and computational processing unit, which can be adapted to and reversibly attached to multiple models of binocular indirect ophthalmoscopes (“BIOs”), enabling simultaneous or time-delayed viewing and collaborative review of photographs or videos from an eye examination. The invention also claims a method for photographing and integrating information associated with the images, videos, or other data generated from the eye examination.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application relies on the disclosures of and claims priorityto and the benefit of the filing date of U.S. Provisional ApplicationNo. 62/456,630, filed Feb. 8, 2017. The disclosures of that applicationare hereby incorporated by reference herein in their entireties.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention is directed to a medical imaging device attachmentwith onboard sensor array and computational processing unit, which canbe adapted to and reversibly attached to multiple models of binocularindirect ophthalmoscopes (“BIOs”), enabling enhanced diagnosticcapabilities to ophthalmologists and optometrists beyond the traditionalmanual ophthalmic examination, such as wireless automatic capture andtransmission of high-fidelity images directly from the perspective of auser performing an eye examination; while allowing the unimpaired, fulluse of the examination instrument via a custom form-fitted mechanicaland optical design; and enabling simultaneous or time-delayed viewingand collaborative review of photographs or videos from said eyeexamination. The invention also includes an integrated system foronboard detection and enhancement of clinical imagery with ambientexamination-related feedback to the user via visual and non-visualinteractive notifications to aid in the diagnostic examination, as wellas the coordinated collection, transmission, management, and maintenanceof imaging and related metadata from ophthalmic examinations, andadditionally allows for multi-user collaboration generated by one ormore device(s) or networks of devices and multiple users.

Description of the Related Art

The state of the art prior to the invention taught herein was for manualbinocular ophthalmoscopes and handheld lenses to view the human eye by apractitioner performing an eye examination, specifically a large portionof the fundus (i.e., the retina, vitreous, and optic nerve). Given theoptical properties of indirect ophthalmoscopy, the examiner is requiredto mentally invert the observed retinal image upside down and backwardsto determine the correct orientation and location of observed findings,with rapidly-shifting focus, significant glare, subtle tilting of thehandheld lens, and small shifts in patient gaze and lighting conditionsoften resulting in a fleeting, limited, and distorted views of thefundus and repeated, lengthy, and uncomfortable examinations for thepatients in order to verify possible clinical observations throughpharmacologically dilated pupils with intensely bright light from theophthalmoscope. These prior art instruments, moreover, were purelyoptical, manual instruments, and unable to capture photographs or videosin real-time during the patient examination. Recording clinicalobservations into the patient medical record with such prior artinstruments requires the examiner to manually draw an artisticrepresentation of findings as they recall their appearance (whileinverting and reversing the optical appearance of findings through theinstrument), or describing such observations in text form. This leads topoor quality of care, due to the inherent limitations of the traditionalfundus exam as discussed above, which require technical staffavailability and separate imaging devices typically located outside thedoctor's examination room (which may or may not be readily available) inorder to conduct clinical photography of the eye.

More recently, a wired video indirect ophthalmoscope has been describedand produced, allowing for digital transmission of photographsconcurrent with the patient examination. However, in practice, theseinstruments typically require two people to conduct such operation,including manually focusing and aiming the onboard camera to betteralign the imaged area with the doctor's view; and managing a wired USBconnection to a computer device as well as a separate wired connectionto a footswitch controller to trigger image or video capture (thustethering the physician to both computer and footswitch, and posing ausability challenge and potential safety hazard in a dynamic clinicalexamination which requires significant movement and repositioning of theexaminer fully around on both sides of the patient to examine both eyesin all gazes of view); along with requiring the simultaneous operationof image management or video editing software with the examiner andassistant having to refer dynamically to video output on a separatescreen while the field of view of the examiner is largely taken up bythe ophthalmoscope's viewing eyepieces, all greatly diminishing theusability and practicability in practice during the patient examination,particularly in the common circumstances in which such trainedassistants are not available during the fundus examination. The inherentusability concerns and additional personnel requirements of such systemshave greatly hindered such video ophthalmoscopes' usage and spread ineye care and limited the public health benefits of concurrent digitaldocumentation of eye examinations. Additionally, the optical arrangementof key components in such prior devices use two or more triangularreflecting mirror blocks enclosed in the instrument straddling adjacentto the central visual axis of the instrument's optical system, incoordination with a centrally-positioned beamsplitter(partially-reflective mirror) in between the two triangular mirrorblocks. The centrally-positioned beamsplitter or prism (such as apentaprism) reflects the central imagery towards a centrally-positionedcamera, but the two adjacent triangular mirror blocks reflect adjacentimagery towards each respective eyepiece. As such, the optical systembears significant complexity of design, and importantly, thecentrally-positioned beamsplitter or prism may partially or totallyocclude portions of the imagery relative to the laterally-locatedoptical pathway to the instrument eyepieces, and cannot in allcircumstances ensure good correspondence between camera and examinerviews. As a result, this requires additional mechanical elements to moreclosely align the camera view with the examiner's view. However, theexaminer's view is nearly totally occluded by the instrument eyepieces.Additionally, in many instances, the eyepieces include low-plusspherical lenses (such as +1.50 or +2.00 diopters) to allow a presbyopicexaminer to comfortably view the aerial image of the examined retinathrough the handheld lens used in indirect ophthalmoscopy. In thesesituations, switching one's view through the instrument eyepieces to anoff-instrument display screen rapidly to guide the assistant orindependently attempt to adjust instrument controls to improvedevice-instrument correspondence, adjust video or image recording orplayback using onscreen controls, or interact with software userinterface elements on an external or even adjacent display may beuncomfortable or disorienting for the examiner. As such, the examiner ofsuch instruments requires an assistant to manually adjust the cameraview using such additional mechanical adjustments such as dials orlevers using an off screen display, close communication with theexaminer, and a process of trial and error to minimize the discrepanciesbetween the examiner and camera views. Furthermore, the mechanics andoptical elements of prior art devices, including the imagingcapabilities, are fully enclosed in the ophthalmoscope, requiring theuser to purchase an entirely new ophthalmoscope machine with suchcapabilities to conduct clinical examinations with the option of digitalphotography/videography, therefore further limiting the practicabilityof greater adoption, because the majority of users already havepurchased and are trained and independently using traditional,non-digital imaging ophthalmoscopes in their practices. Also, in otherdesigns, the prior described instruments require partial or completedisassembly or other modifications of the external enclosure of theexisting ophthalmoscope in order to accommodate a proposed retrofittedattachment camera with a securely aligned mounting assembly, or requireadditional mounting hardware and special tools for installation such asmounting pins, screws, or other invasive elements, which may beimpractical for most users, may damage the users' purchased instruments,and may potentially void the warranties or service agreements for users'existing instrument purchases. Additionally, the great expense ofprevious video ophthalmoscopes (on a par with benchtop fundus cameras)typically only allows for installation in one examination lane for atypical ophthalmic practice, not only providing inferior quality imagesthan a benchtop fundus camera due to inherent limitations in indirectophthalmoscopic image quality as well as previously-discussedlimitations, but also requiring the physician to capture all images(technicians are typically not trained in indirect ophthalmoscopy) andrepresenting a bottleneck in the clinical workflow of a modernophthalmology or optometry practice, as all patients in a typicalmulti-lane setting to be imaged would have to be redirected to aspecific examination lane following a traditional examination.

The remaining alternatives other than an ophthalmoscope with an embeddedcamera in order to obtain clinical photography are in using traditionalbenchtop fundus cameras and scanning laser ophthalmoscopes, both ofwhich are expensive, bulky, and require the use of a trained retinalphotographer. Nonmydriatic fundus cameras and scanning laserophthalmoscopes, which do not require the pharmacologic dilation of thepatient's pupil, can incentivize physicians to not dilate theirpatients, and thus not examine the retinal far periphery; however,nonmydriatic fundus photographs routinely produce image artifacts whichcan be read as false positives for alarming features such as retinaldetachments and posterior segment tumors, while being unable to examinethe retinal far periphery, disincentivizing eye physicians fromconducting a complete dilated examination of the ophthalmic fundus andleading to inferior quality care. Additionally, communication gapsbetween eye physicians and retinal photographers routinely limit thequality and accuracy of images actually obtained in clinical practice,and staffing and patient scheduling gaps limit whether photography canbe conducted at all using such technician-performed techniques.Smartphone and other mobile camera systems are alternatives to the fixedfundus camera, but face relatively poor patient acceptance, are oftencumbersome to use (particularly in focus and exposure control), requireyet another proprietary device to capture photographs redundant to theretinal examination, and typically also cannot adequately capture theretinal periphery. The present invention promotes the gold standardtechnique for retinal examination and the only technique allowing fordynamic examination of the full retinal periphery (indirectophthalmoscopy), as recommended by both optometric and ophthalmologyprofessional bodies, and the added benefit of simultaneous capture andredisplay of fundus photography and video, using the same existingexamination instruments doctors already possess, are trained upon, andare comfortable using, allowing for augmented retinal examinationwithout introducing a separate step in clinical image acquisition beyondthe doctor's own examination process.

Moreover, neither of the aforementioned options—traditional benchtopfundus cameras and scanning laser ophthalmoscopes—are capable of imagingthe full periphery of the retina where many retinal detachments andseveral kinds of posterior segment tumors originate—this is somethingonly indirect ophthalmoscopes are currently capable of fully viewing(e.g., the retinal far periphery), in combination with dynamic fundusexamination procedures familiar to ophthalmologists and optometrists,such as scleral depressed examination techniques, which can be performedin concert with other diagnostic and therapeutic maneuvers such asfluorescein angiography, scleral indentation, and indirect laserphotocoagulation.

SUMMARY OF THE INVENTION

A core problem solved by the current invention is in gaining the abilityto seamlessly take clinical photographs with a high degree of fidelityfrom the ophthalmic fundus examination—in a preferred embodiment, of thepharmacologically-dilated eye of a patient—taken by the user using thestandard examination instruments they have already purchased and havebeen trained to properly use in their routine clinical practice, withthe free and full use of all existing settings and onboard BIOinstrument illumination and view-related controls, with minimal or nomodifications or alterations necessary in the examiner's routineexamination technique, and no damage or disassembly of their examinationequipment necessary for the reversible mounting and secure positioningof the camera and computational apparatus. The current invention alsoencourages the gold-standard examination technique of theretina—indirect ophthalmoscopy—and makes possible seamless wirelesstransmission of clinical photographs and videos from the clinicalexamination to other viewers such as students or other practitioners,for training, patient care, clinical research, telemedicine, andfeedback during or after an examination, via integrated onboard andoff-device networked computational and software elements. Thistransforms the traditional manual fundus examination into an augmentedimage-guided examination, allowing the doctor and patient the benefitsof clinical photographic documentation and enhancement, while optimizingworkflows.

Improvements over the prior art include, but are not limited to, theability for the user to simultaneously capture ophthalmic featuresmanually; automatic device capture of images integrating onboardintegrated sensors, computational processing capabilities, andtightly-integrated on-device and off-device algorithmic processing ofimagery; allowing for feature recognition of the eye and ocular featuresof interest; and/or automatically montaging multiple overlapping imagesto more broadly map and redisplay the ocular features via networkedsoftware programs; all included in a very small, integrated form factorsuch that the claimed device can be self-contained and reversiblymounted by the user on a pre-existing ophthalmoscope without specializedtraining or tools—and without procuring a new ophthalmoscope, andwithout damaging or directly modifying the preexisting instrument. Inother words, the presently taught device allows an existingophthalmoscope to be retrofitted with a mobile embedded imaging systemto make it an imaging device with wireless capture and transmissioncapabilities, in preferred embodiments, and it can be implemented by theuser without having to obtain a new ophthalmoscope. The device taughtherein improves upon prior optical design by simplifying the opticalsystem by requiring only one centrally-positioned triangular reflectingmirror block (as is typically found inside the optical system of mostBIOS) and an onboard linear plate beamsplitter, not requiring a prism orlaterally-located mirror blocks; as opposed to two or morelaterally-positioned mirror blocks in coordination with a prism (such asa pentaprism) or centrally-positioned or laterally-positionedbeamsplitter, which ordinarily may totally or partially occlude theoptical pathway to the instrument eyepieces. Such prior configurationsintroduce significant device complexity to the mechanical design andopportunities for distortion of imagery, and cannot ensure consistentcorrespondence between camera and examiner views in many examinationscenarios. The optical system here described allows for a greaterfidelity of correspondence between onboard camera and examiner viewsover a much-greater breadth of examination scenarios. In one embodiment,a mechanical adjustment lever allows for customization of the opticalsystem by tilting, in a coplanar relationship, the beamsplitter mirrorand embedded camera assembly, to permit a greater range of examinerwearing patterns of the instrument, such as physicians who wear theinstrument at a significant downward tilt to examine patientssignificantly below their eye level, or physicians who wear spectacles.In another embodiment, the use of a wider field of view camera lens,along with a high-resolution camera sensor, may be used to customize theviewing angle of the captured imagery to the preferred instrumentviewing position of individual users, by setting the desired viewingregion by cropping out extraneous imagery via a software interface. Inone instance, this software-enabled field of view image crop control maybe used in combination with an initial user calibration procedure using,but not requiring, the use of a standardized target to allow forautomatic configuration of the camera view—without requiring interactionwith the onboard mechanical device controls such as adjustment levers ordials.

Beyond its optical properties, the device and integrated systemdescribed also allow for subsequent review and enhancement of ocularimagery beyond the range of view ordinarily visible or accessible to theotherwise unaided eye of the examiner using their existing manual BIOinstrument. The apparatus thus permits the routine use of image-guidedaugmented examination for improved patient care, enhanced detection anddocumentation of clinical pathology, and serial comparison of follow-upfindings, as well as improved practice efficiencies by the simultaneousdual use of BIO instruments for both examination and enhanced clinicalimaging purposes.

The design allows for transmission of data, an image trigger thatautomatically captures when certain features are present in theophthalmoscope's viewfinder, manual focus, closed-loop and open-loopautofocus, and other optical and sensor-assisted algorithmic techniquessuch as focus stacking, software-selectable focus planes, expanded depthof field imaging, and region of interest (ROI)-based focus and exposurecontrols to ensure crisp focus and exposure without or with minimal userintervention in routine clinical examination settings, imageenhancement, automatic montaging to see more complete pictures of, forexample, the retina, and annotation of findings. Focus stacking issimilar to expanded depth of field imaging; also known as focal planemerging and z-stacking or focus blending; it is a digital imageprocessing technique which combines multiple images taken at differentfocus distances to give a resulting image with a greater depth of field(DOF) than any of the individual source images. Focus stacking can beused in any situation where individual images have a very shallow depthof field, such as encountered in indirect ophthalmoscopy. Unlikeexisting prior art, in one embodiment, no display screen (eitheron-device or off-device) is required to be referenced during examinationsessions for clinical findings to be imaged with high fidelity, asfocus, exposure, and field of view are all faithfully maintained by thedesign of the device. Though one prior art system describes review ofimages on the device itself on a display screen either attached orlocated adjacent to the viewpiece or handheld, given that the indirectophthalmoscope viewpieces themselves inherently occlude and take up mostof the examiner's field of view, referring to any external displayscreens while actively examining a patient is quite difficult inpractice. For this reason, no external or on-device display screen isrequired in this device design, and device notifications are designed tooccur “ambiently,” that is, without obstructing or minimally obstructingthe examiner's view, not requiring the examiner to alter their gazeduring the clinical examination and still receive examination- anddevice status-related interactive notifications. All of thesecapabilities are enabled by the same device mounted on an existinginstrument with minimal user intervention. Ambient notificationsinclude, but are not limited to, light feedback, sound feedback, hapticfeedback, touch feedback, vibrations, buzzes, clicks, noises, beeps, orspeech, or any other notification or feedback that does not interferewith the examiner's examination of the patient's eye, such as byoccluding or obstructing the examiner's view or otherwise distractingthe examiner or limiting the examiner's movement.

Software-selectable focusing planes are an algorithmic and optical setof techniques in which one or more focal planes can be selected from oneor more electronically-controlled cameras, lenses, or image sensors, inwhich, in a preferred embodiment, an electronically-adjustable focallength can be selected or tuned electronically for anelectronically-tunable lens, or desired focal length can be selectedfrom an array of cameras adjusted to a variety of focal lengths, tooptimize focus of the images/video captured by the device to the workingdistance between instrument and handheld condensing lens used by theexaminer. In a preferred embodiment, these functions can be performedwithout the user having to resort to manual adjustment to mechanicallevers, dials, or switches, to improve ergonomics, enhance operationalworkflow, and minimize or eliminate the need for an assistant to adjustthe focus manually for individual users or examination parameters inwhich the lens may be held at a shorter or longer distance than typical.Additionally, the use of such techniques may enable, in one aspect,greater functionality of the device beyond the typical usage, in whichthe user can easily switch between a variant focal length or combinationof focal lengths to enable in-focus image/video capture of structures inthe anterior segment of the eye or around the eye, with or without theneed for an additional examination lens, and then switch back to normalimaging modes to focus upon the aerial image of posterior segment ocularstructures as imaged through the handheld condensing lens.

Expanded depth-of-field imaging is an algorithmic and optical set oftechniques in which one or more electronically-controlled cameras,lenses, or image sensors with different imaging characteristics (mosttypically, cameras with varying focal lengths and f-numbers/apertures)are used in combination to algorithmically join multiple images capturedat varying focal lengths into a composite image or images capturing ahigher depth of field image with higher image quality given limited orvarying ambient light than would ordinarily be captured by using onecamera or sensor in isolation.

Region of interest-based imaging is an algorithmic and optical set oftechniques in which certain preferred areas of the image(s) or videocaptured by the image sensor(s) to set global imaging settings(including, but not limited to, focus, exposure, white balance, andimage trigger control) can be reprogrammably controlled by the user viaa text- or graphical user interface on a separate networked computingdevice, or pre-set to minimize the need for user intervention. Incertain aspects, additional image processing and computational imagerecognition techniques may be used including, but not limited to,recognition of certain ocular structures or abnormal or normal clinicalfindings, to dynamically set or change associated imaging parameterswith minimal need for user intervention.

Use of onboard communications/telemetry transmission embedded in thedevice allows for multiple options for off-device transfer of data forreviewing, filing, and displaying clinical images and video. Forexample, this includes quick image review on a mobile device,simultaneous re-display on a tethered or untethered, networked videomonitor (e.g., by Bluetooth, WiFi, radio frequency, or Internetconnection), remote review by a supervising physician or other permittedthird party, remote analytics data collection, concurrent sharing ofclinical images and video with consulting practitioners (e.g.,specialists), and seamless generation of imagery and clinical metadatawhich can be scrubbed (manually or automatically/algorithmically) ofprotected health information and patient identifiers to quickly generatelarge datasets of clinical imagery and physician practice patterns,suitable for data science applications for continuous qualityimprovement, such as machine learning, artificial intelligence andprocess automation in healthcare applications, and clinical researchpurposes. The device and integrated system can enable wired or wirelessdirect transmission to electronic medical record systems andpoint-of-care billing by an examining practitioner, and concurrentpoint-of-care sharing of clinical images/video to patient and familiesof patients for medical education and facilitating continuity of care.Onboard networked computational processing capabilities allow, in oneembodiment, the off-device automatic recognition of or manual accountlogin of authorized users of the device, automating the process oflinking examining physicians to clinical imagery data for examinedpatients, examination sessions, and location. In one instance, suchnetworked sensors would allow for automatic user recognition,authentication, and tagging of generated imagery in an examinationsession via an NFC or other wireless token carried by an authorizeduser, a securely paired smartphone, or via a QR code identification tagthat, in one embodiment, would be recognized via computer visionalgorithms, using the onboard camera or cameras and embeddedmicroprocessor. Integration of onboard sensors, networked antennas, andembedded camera or cameras along with on-device and networked softwarecan allow for simple, contactless wireless network configuration andcoordination of multiple instances of the taught device within aclinical setting. The tight integration of the taught device hardware toreprogrammable on-device software and off-device software allows for agreatly improved user experience, improves usability, and enables abetter-quality clinical examination beyond prior art hardware-onlyapproaches. The device and system allows for augmented clinicalexamination facilitating review of images/videos by the practitionerwithout unnecessary discomfort to the patient following dilatedexamination, as the current clinical practice necessary to producefundus photography on a routine basis usually requires repeat serialflash photography through the dilated pupil subsequent to or prior tothe separate DFE (dilated funduscopic examination, which is conductedseparately by the examining ophthalmologist or optometrist), asconducted using a dedicated large benchtop fundus camera by anophthalmic technician in typically a physically separate clinical areafrom the examiner's medical examination room. As outlined earlier,practitioners performing the examination must remember what they see andthen either verbally describe or draw a crude picture (either by hand,or using a rudimentary painting application) of what they observed basedon their memory. The device will decrease the time necessary for suchexaminations (which are typically both inconvenient and uncomfortablefor the patient), wait time for patients currently forced to wait forthe availability of examining physicians as well as trained techniciansto capture fundus photographs, and substantially reduce the possibilityof human error due to potential for error inherent in clinicalexamination and documentation methods reliant solely upon human memoryand physician documentation.

In one aspect, for those models of BIO instruments with LIO (laserindirect ophthalmoscope) functionality, the attachment/adapter systemdescribed allows simultaneous capture and remote redisplay of imagery,as well as feature recognition/augmented examination and treatment (suchas guiding the ophthalmologist as to the size and region of the fundusto be treated with laser photocoagulation, the region already treated,and tracking stability vs. progression of pathology beyond previousregions of laser treatment. The system also allows for remotecollaboration (real-time or delayed collaboration) for laser surgicalprocedures (in one application, for redisplay of barrier laserphotocoagulation of retinal breaks to help guide the laser surgicalprocedure) for clinical documentation, telemedicine consultation, andmedical education purposes.

In another aspect, the device can be reversibly attached to an existingophthalmoscope by a practitioner or other untrained individual. In oneembodiment, the device comprises a self-contained system comprising oneor more cameras, a beam splitter (e.g., a partially reflective mirror)that is at a 45-degree angle to the incoming optical rays and theexaminer so that the image being viewed by the examiner is the same ornearly the same as the image viewed by the camera, one or more antennasfor wirelessly transmitting data to connect the device with one or moreexternal devices (e.g., a computing device), user accessible memory, anda battery. The device also comprises, in one embodiment, a custom-fitbracket to attach the device to an ophthalmoscope. In one aspect, thedevice is positioned vis-à-vis the ophthalmoscope with the bracket (inone aspect, with adjustability) to allow the optics of the beam splitterand camera assembly to fit flush (or close to flush) and aligned (orclose to aligned) with the external view port of the ophthalmoscope andaligned (or partially aligned) with the examiner's view; in thisembodiment, the camera or cameras are located in between the eyepiecesof the examination instrument and unlike in some prior art designs, isnot dependent upon or limited by the field of view from only one of thebinocular eyepieces. In one embodiment, two or more cameras may be usedto enable a three-dimensional or high-dynamic range view. One or morebrackets may be included to adapt to several types of ophthalmoscopemodels. The design of these brackets allows for full (or close to full)adjustment and operation of all BIO controls and unoccluded (or nearlyunoccluded) passage and adjustment of optical imagery to the examiner aswell as illumination from the BIO instrument to the patient's eye underexamination. The device design does not require any onboard illuminationsource, but rather allows unobstructed (or nearly unobstructed)transmission of light and manipulation of the BIO illumination source.In one embodiment, the design of the brackets, embedded camera orcameras, and optical system relative to the examination viewing windowat the front of the ophthalmoscope further allows for selectable ortunable alignment of the optical system and field of view by the user toaccommodate ergonomic usability by the user, accommodate eyeglasscorrection by the examiner, and tilt and positioning of theophthalmoscope on the examiner's head while reproducing imagery with ahigh degree of fidelity to the examiner's view. The battery and powermanagement system may be tethered by wire or incorporated in the deviceenclosure fully, and may be user accessible for easy replacement whilein use (“hot swapping”) for extended examination periods betweencharging. The device also comprises a microprocessor, with connectedcommunications antenna system consisting of one or more types ofwireless antennas, as well as an integrated sensor array. In one aspect,the processor elements may comprise a reprogrammable system on a chip(SOC) or system on module (SOM), with one or more custom carrier PCBs(printed circuit boards), running the on-device software taught herein,bidirectionally communicating with off-device networked elements andsoftware, and comprising the non-optical functions of the device andintegrated system described. The reprogrammability of the system incombination with bidirectional networked interface with other onboard oroff-device software elements and external computing devices representadditional substantial improvements to the prior art and enable a hostof additional functionality to the integrated system, such as hereindescribed.

Image capture may be accomplished by an external foot switch, which maybe wired or wirelessly connected to the device, such as through a pairedantenna. The image may be captured by remote image trigger either by ahardware device controller, or an off-device networked software controlpaired to the specific instrument and examination session. For example,in a clinical education setting, a teacher may capture an image while astudent is using the device to examine a retina to capture a desiredimage or images of a specific feature of interest. Also, using an imagepreprocessor, sometimes in coordination with the processor, and aminiaturized camera, in combination and connected with a second or morecomputing device(s) (e.g., a mobile phone or EMR computer), auto captureis enabled along with auto enhancement, and auto-analysis, using machinelearning and computer vision techniques. Imagery (still and/or video)may be processed either on-device, or by using post-processingtechniques on a more robust connected computing device (such as, but notlimited to, mobile smartphone, networked server, cloud-connected arrayof servers, or desktop or portable computer), so as to decrease the bulkand power requirements of the described device. This innovation can beused to, in one embodiment, automatically generate a high-quality imagelibrary of the examination and of multiple serial clinical encountersover time for rapid cross-comparison and cross-reference of the imageswhile partially or wholly automating much of the ordinarilylabor-intensive process of manual association and filing ofexamination-specific elements or metadata such as the examiningphysician, patient, and date of service.

In the optical system of the device, in addition to the beam splitter,optical glare reduction techniques may be utilized, such as in oneembodiment, the use of one or more linearly polarized plates withvariable light transmission (e.g., a linear plate polarizer), which maybe used to polarize the outgoing light from the indirect ophthalmoscopeillumination source, as well as one or more additional linearlypolarized plates located prior to the camera image plane (polarizing theincoming light beams comprising the image being captured by the embeddedminiaturized camera system), to remove glare and other stray reflectivedistractions from the image optically, thereby resulting in a higherquality clinical image. These optical glare reduction techniques enabledby the physical design of the device optical system may be used inmultiple combinations with, or without, the use of dynamic algorithmicimage processing techniques to reduce glare, which are enabled by, inone embodiment, the onboard microprocessor, or, in another embodiment,off-device software hosted remotely, such as in the cloud, or on a localnetworked computing device.

In one embodiment, auto-capture comprises automatic algorithmicdetection as to whether a retina and/or optic nerve (or other fundusfeature) is in view and in focus. If that is the case, the camera willautomatically take a picture with no user intervention. This techniquecan aid in the usability of the instrument, in which there may be adelay between the user observing a feature of interest, small movementsof the eye under examination, and capture of imagery using a manualtrigger, resulting of non-capture of the desired imagery. Thiscircumstance can be further minimized by, in one embodiment, use ofconstant automatic rapid capture of a series of images heldalgorithmically in an onboard image storage buffer in the computationalsystem, such that after a manual image trigger signal generated by theuser, the desired image or images of interest can be selectedsubsequently upon review via networked off-device software, from anautomatically generated library of session images for later storage andretrieval. Auto-enhancement means that image or video capture will beautomatically enhanced, meaning the image will automatically normalizesuch features as contrast and color, and minimize glare from reflectedillumination using dynamic algorithmic image processing techniques, andincludes, but is not limited to, digital sharpening, contrastenhancement, and removal of glare and occluding elements, improving thequality of imagery beyond what the user is able to ordinarily see withan un-augmented examination instrument, and improving the ease ofcross-comparison between individual images and between clinical imagingsessions. In one aspect, the image will be centered, a standardized cropor border will be placed around the picture, and image orientation markswill be added algorithmically via on-device or off-device software.

Auto-analysis includes, but is not limited to, automatic algorithmicdetection, flagging, and measurement of clinical features of interest,and comparison with prior detected features for evidence of clinicalstability versus change, as well as comparison with reference databasesof normal and abnormal clinical features, using a variety of techniquesincluding, but not limited to, computer vision, deep learning, andartificial intelligence. In one aspect, auto-analysis will occur usingconnected software to correlate clinical images with an external libraryor set of algorithms determining attributes such as which eye is beingexamined, or flagging optic nerve and retinal periphery, noting abnormalfeatures detected by the system, all of which may aid in clinicalexamination upon review of the image(s). Though such techniques havebeen used for analysis of ophthalmic imagery using traditional funduscameras separate from the eye physician's examination, in thisembodiment, via close integration with a standard ophthalmic examinationinstrument—in this taught embodiment, the indirectophthalmoscope—algorithmic image enhancement and machine vision would bepossible in real-time or close to real-time directly from the clinicalexamination session itself, allowing the benefits of algorithmic imageanalysis of ophthalmic imagery in clinical decision-making directly atthe point of care, thus greatly improving practice efficiency and theclinical potential of any one examination and clinical encounter. Autoanalysis may also enable redisplaying image flags oralgorithmically/computationally-generated metadata in multiple formats,such as, but not limited to, text or annotated images and video. In oneembodiment, auto analysis can display its output by electronicallytransmitting metadata and clinical imagery to a connected EMR/EHR(Electronic Medical Record/Electronic Health Record) system or aseparate connected computing device or application linked to a patient'selectronic chart. In another embodiment, redisplay of auto analysisresults can be accomplished by superimposing automatically-generatedtags and/or graphical overlays illustrating areas of concern upon thecaptured imagery. Using pre- or post-image processing, the images takenduring the examination process or generated from video capture, willautomatically join photographs capturing adjacent regions of the fundus,synthesizing a montage map of the patient's fundus automatically withminimal or no user intervention, enabling cross-comparison of imagesbetween patient examinations and between patients. Such cross-comparisoncan also be conducted, in one embodiment, by quick point-of-carecross-reference to a normative or pathologic database of fundus imagery(still or video) to enable immediate or close to immediate reference ofpatient pathology to an external image library for augmented examinationenabling substantially enhanced clinical utility of the dilated fundusexamination by the use of augmented examination technology. This willmake the examination shorter and more comfortable to the patient, whilepermitting the practitioner a longer time to subsequently study thefundus by and through the images captured by the device and integratedsystem, while allowing for an enhanced level of detail, image quality,and diagnostic decision-making aids beyond which would be ordinarilypossible in un-augmented ophthalmoscopy.

The device also incorporates security features to maintain patientconfidentiality, system integrity, and integration of the device,integrated system, and other connected devices into an existing secureinformation technology network for use in a clinical setting. Videos,images, and clinical metadata, clinical data, user account information,and any other associated telemetry data pertinent to the operation ofthe device or system(s) used in the clinical data network may beencrypted by the device itself, allowing for secure transmission of datato a trusted, approved user or group of users, and allowing for ahierarchical data trail to be generated for access and manipulation ofclinical data. Physical tokens, passcodes, or connected “trusteddevices” (in one embodiment, a trusted device is a device that has beensecurely authenticated to an authorized user) can be used, in oneinstance, in combination with onboard telemetry such as, but not limitedto, networked antennas to automatically detect the presence, absence,and use of the system by a “trusted team member” (in one embodiment, atrusted team member is a user authorized to conduct a specifiedexamination or procedure and authenticated within a particular clinicalcare provider system) and appropriately tag and file generated imageryand metadata with a hierarchical audit trail to maintain data integrity,automate the appropriate tagging and filing of generated clinicalimagery and documentation, and maintain clinical data in compliance withrelevant regulatory protocols for protected health information, as wellas for clinical research data applications.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate certain aspects of some of theembodiments of the present invention, and should not be used to limit ordefine the invention. Together with the written description the drawingsserve to explain certain principles of the invention.

FIG. 1 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, including the BIO, mounting bracket, andoptional optical cartridge.

FIG. 2 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, including the BIO, mounting bracket, andoptional optical cartridge, with certain components disconnected andshown separately. The cartridge is optional as a separate or separablecomponent. The mounting bracket and cartridge can all be one piece, suchas a singular or modular molded, formed, casted, constructed, extruded,pultruded, shaped, connected, united, associated, coupled, joined,affixed, linked, bonded, fastened, appended, glued, attached, or 3-Dprinted piece.

FIG. 3. is a flowchart showing possible software layers used to shareand store captured and processed images.

FIG. 4 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, specifically the optionally separableoptical cartridge.

FIG. 5 is a schematic diagram of a depiction of one possible embodimentof the device taught herein.

FIG. 6 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, including the BIO, mounting bracket, andoptional optical cartridge.

FIG. 7 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, including the BIO, mounting bracket, andoptional optical cartridge.

FIG. 8 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, including the BIO, mounting bracket, andoptional optical cartridge.

FIG. 9 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, including the BIO, mounting bracket, andoptional optical cartridge.

FIG. 10 is a schematic diagram of a depiction of one possible embodimentof the device taught herein.

FIG. 11 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, including the BIO, mounting bracket, andoptional optical cartridge.

FIG. 12 is a schematic diagram of a depiction of one possible embodimentof the device taught herein, including the BIO, mounting bracket, andoptional optical cartridge.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS OF THE INVENTION

The present invention has been described with reference to particularembodiments having various features. It will be apparent to thoseskilled in the art that various modifications and variations can be madein the practice of the present invention without departing from thescope or spirit of the invention. One skilled in the art will recognizethat these features may be used singularly or in any combination basedon the requirements and specifications of a given application or design.Embodiments comprising various features may also consist of or consistessentially of those various features. Other embodiments of theinvention will be apparent to those skilled in the art fromconsideration of the specification and practice of the invention. Thedescription of the invention provided is merely exemplary in nature and,thus, variations that do not depart from the essence of the inventionare intended to be within the scope of the invention. All referencescited in this specification are hereby incorporated by reference intheir entireties.

The hardware and system taught herein allows for the ability tosemi-automatically or automatically generate fundus photographs and alive digital redisplay of the clinician's view (though not requiring anon-screen image for accurate image capture) during the conventionalworkflow of the eye exam itself, without requiring a trained technicianor separate machine to do so. This allows for each patient examinationlane to serve a dual purpose, as a combination patient examination andfundus photography/documentation station. The use of onboard andconnected computing elements allows for the rapid automation of severalelements previously requiring a dedicated technician or assistant, andphysically separate image acquisition system (such as benchtop funduscamera), in order to acquire fundus photography/clinical documentationof the posterior segment examination of the eye in addition to theclinical examination of the eye by a qualified eye care physician. Theability to conduct a retinal exam and routinely obtain a concurrentimage capture session and also enable a simultaneous, real-time,store-and-forward, or subsequent image review process, with no or almostno additional requirement for a technician or assistant to conduct thephotography, in addition to increasing clinical practice efficiencies,should enhance the clinical review process, accuracy, and interpretationof the generated images, which will increase the chances of detectingearly, subtle disease symptoms, such as early non-proliferative diabeticretinopathy (NPDR), or optic disc changes in the evaluation of glaucoma,especially compared to current examination techniques, such as using themanual BIO examination instrument alone.

Moreover, this apparatus and system makes for the ability to share,export, send, collaboratively tag, annotate, engage in remoteconsultation with colleagues, or otherwise allow for viewing outside theophthalmoscope viewfinder the images captured during or after thepatient encounter within certain medical practices or academia,including viewing remotely (e.g., telemedicine). The hardware attachment(and software interaction/system) does not interfere or minimallyinterferes in the examination process of the patient; for example, thehardware attachment should not occlude, or minimally occlude, the viewof the examiner, nor should it significantly occlude the BIOillumination source (e.g., halogen or LED light beam). In the case of abeamsplitter as part of the apparatus, the apparent brightness to theuser can be lowered or raised, but not dramatically. In other aspects,the split is 70:30 (examiner:camera light intensity ratio), which inpractice is clinically indistinguishable or minimally distinguishablefrom the traditional examination and maintains light intensity of BIOillumination well within standard illumination thresholds. In oneaspect, unlike other prior art systems, the apparatus and system do notrequire the examiner to use a screen other than what can be seen withinthe scope to verify images while capturing such images.

Capturing images will be aided by multiple redundancies. In one aspect,a redundancy is automatically built in to the system; for example, ifthe image capture does not work as programmed, the user may still usethe BIO manually as they would normally. In another aspect, if onesystem does not work as expected, or is inappropriate for a particularsetting, an alternative method will be substituted for the same task(e.g., wireless image transfer from the device's onboard electronicstorage media may not be appropriate in certain medical institutions, inwhich case a USB or SD card manual image transfer may be used instead orin addition to wireless transmission). Similarly, in some aspects,redundancies may include certain wireless networking technologies suchas, in certain aspects, Bluetooth or one or more or variations of WiFi,as well as printing images, mobile application, web-based application,computer-based application, cloud-based application, hardware-basedapplication, etc.

Image and video capture can be triggered by multiple methods includingbut not limited to via a remote app (such as on a cell phone, tablet, orcomputer), automatically, by a handswitch or footswitch, or by usingon-device or off-device microphones and voice recognition technologyconducted on-device or off-device to capture, switch imaging modes,annotate imagery, or otherwise control various elements of the deviceand integrated augmented examination system, without interfering with,or minimally interfering with, the usual indirect ophthalmologyexamination procedures conducted by the user.

In one aspect, the hardware comprises six optionally separate physicalcomponents, all of which are physically attached to a BIO instrument.Those six components comprise:

-   -   The camera attachment (in one aspect, wired to a reprogrammable        microprocessor board);    -   Reversible mounting bracket assembly;    -   Optical beamsplitter attachment, optical and illumination        transmission window, and optional linear plate polarizing        element or elements to reduce glare;    -   Wired or wireless foot-pedal image trigger or other means of        triggering image capture;    -   Embedded microprocessor, which, in one aspect, comprises a        microcomputer and connected wireless antennas and sensor array        integrated in the device enclosure, such as a wireless (in one        embodiment, Bluetooth) connection with networked computing        devices in “listening” mode awaiting input from recognized        networked devices, for use cases such as, but not limited to,        image review on an external networked computing device, or        software-based store-and-forward image sharing; and    -   A power source, such as an AC (alternating current) power        adapter or battery, which may be connected to a wired or        wireless (in one embodiment, induction coil-based) charging        system or “hot-swappable” battery array system.

Regarding the camera attachment in the main device enclosure, it mayexhibit exact focus correspondence (to an in-focus image manuallygenerated by a user of the indirect ophthalmoscope using a handheldcondensing lens and standard indirect ophthalmoscopy examinationtechniques), replicable in production to parallel rays and correspondingfocal length to capture reflected rays from the beamsplitter image. Inone aspect, it allows for standardized focal elements of the onboardcamera, e.g., “number of turns” of the camera barrel threads, to achievefocus if using a custom camera mount. In one aspect, focus is defined bylens barrel length and should not “drift” once set; preferably, thefocus ring is not accessible by the user except in a servicingsituation. A variety of hardware- or software-based techniques may beused to achieve consistent focus correspondence between the aerial viewof the retina through the user's handheld condensing lens, theexaminer's view through the BIO instrument eyepieces, and the imagecaptured via the onboard camera, such as a high-depth of field camerasystem, such as, but not limited to, the use of an onboard lenspreferably with aperture greater than f4.0, unlike most embedded camerasystems with narrow depth of field and aperture approximately f2.0.

In one embodiment, there is exact, or almost-exact, alignment betweenthe camera, BIO view through the eyepieces, and device beamsplitter asenabled by precision mechanical design of the device's reversiblemounting bracket mechanism, optical and illumination transmissionwindow, and orientation of optical elements relative to the optical andillumination system enclosed wholly within the BIO instrument, which inmany aspects, typically comprises a single central triangular reflectingmirror block, reflecting imagery laterally (and again, via a second setof two triangular mirror blocks) onward to the BIO instrument eyepieces;and a superiorly located illumination source reflected nearly coaxial tothe optical axis of the BIO instrument via another, superior butcentrally-located, angled mirror. A triangular reflecting mirror blockused within a standard BIO hardware system is known to those of skill inthe art. In certain cases, this may require either separate “wings” froma central hardware attachment module to accommodate secure precisionalignment with different BIO instrument models, versus adjustment knobsor entirely different device enclosure models specifically form-fit tospecific BIO instrument makes or models, if too large a differenceexists in the mechanical design aspects between different BIO instrumentmodels to permit consistent alignment of device elements, opticalelements, and the minimal obstruction of instrument controls by use of a“universal” design of the mounting bracket mechanism.

In one embodiment, the design will use a 2-dimensional imageapproximating an “intermediate” image in between oculars (preferablyseparated by about 8 mm at external viewport, down from ˜60-70 mminterpupillary distance of examiner), by a splitting ray in the middleof the viewport, central location of a partially reflective mirrorpositioned coaxially with the wholly-enclosed optical elements of theBIO instrument being adapted, and the central location of the deviceembedded camera located coaxially with the reflected image. Otherembodiments may split off two image rays to two cameras, one for eachviewing mirror, capturing a one-to-one 3-dimensional view, or to enableother imaging enhancements enabled by a two-camera system, such asenhanced depth of field, level of zoom, increased sharpness, expandeddynamic range, or other optical and/or sensor-based viewing enhancementsof captured imagery, such as red-free fundus photography, fundusautofluorescence (FAF), or fundus angiography (FA) by the use of asecond camera and specially tuned optics (such as using specializedimage gratings or sensors to capture particular wavelengths of lightwith or without the use of intravenously-injected photo-emissive dyessuch as fluorescein dye, useful in clinical examination of the fundus inconditions such as diabetic retinopathy, but traditionally requiringmuch larger dedicated imaging systems and dedicated technicians).

In preferred embodiments, the camera will be neither too large nor tooheavy to significantly impair a typical examination, so as to maintaindesirable user ergonomics during the examination session. If certainembodiments are required to be larger or heavier, the camera and mountedapparatus is designed to balance weight between front and rear sectionsof the camera, taking into account places where a user willtouch/manipulate or where their hair or glasses could become entangledor otherwise affect the clinical examination and/or image capture. In apreferred embodiment, the apparatus taught herein will comprise an “allin one” housing design, incorporating camera, beamsplitter/optics,computing/processing/transmission/sensing means, and battery (or otherpower means (e.g., AC adapter)). In one embodiment, the battery may bemounted externally on the user, such as at the waist (for example on abelt), with a wired attachment to the main instrument-mounted device.

In preferred embodiments, any wires, such as the camera control wire,will not be exposed outside the apparatus, although in some embodimentsthe control wire and/or other wires will be exposed.

The optical system may comprise a separate, air-gapped smallbeamsplitter (partially-reflective mirror) to be attached or alignedover the BIO viewport. The beamsplitter, in one aspect, may be a simple70/30 plate beamsplitter, reflecting a second ray 90° from the opticalpath. The beamsplitter design, rather than a coaxial camera, may be usedin one aspect, having near-100% correspondence between the user'sexamination and the generated images with minimal requirement for theuser to have to manipulate mechanical control levers on the apparatus,or to have to make significant adjustments to achieve user-cameraoptical correspondence before, during, or after the examination intypical use cases. The user may generate focused images (parallel rays)by, most commonly, a handheld 20-diopter or 28-diopter condensing lensas the user has been trained to do in routine indirect ophthalmoscopy,taking into account factors like image tilt, corneal distortion, and apatient's refractive error of the eye examined; when the aerial image ofthe ophthalmic fundus, as viewed through the handheld condensing lens,is in focus for the user, it should be in focus for the camera. The usermay, in one embodiment, calibrate the device focus in a separate set-upprocedure before using the instrument for the first time (in oneembodiment, aided by the use of an infrared-based distance sensor) toaccount for the examiner's own refractive error (such as presbyopia) andallow sharp focus of the device camera upon the aerial image generatedby the handheld examination lens.

In some aspects, ancillary optical systems may be used or included(e.g., condensing lenses to shrink the image, or parabolic mirrors)beyond just the camera/beamsplitter, allowing for furtherminiaturization and an all-in-one design, while also exhibitingtechnical advancements. Image distortion (e.g., using a parabolicmirror) may or may not be unacceptably introduced by such a system; ininstances where a consistent type of optical distortion is produced bysuch additional lenses or mirrors, onboard or off-device dynamicalgorithmic image processing techniques may be used to correct for orreduce distortion in the captured images. In a preferred embodiment, theoptics are designed to remain securely flush and aligned with the BIOeyepiece external viewport, and not occluding or distorting (orminimally occluding or distorting) the view of the examiner, or thelight from the BIO illumination source emitted through the headlampportion of the BIO viewport and passing through the device opticaltransmission window.

The apparatus may be adjusted to “fit” or accommodate an array ofdifferent BIO models. Additional hardware may also be provided, such asan attachment bracket specifically designed for individual BIO makes andmodels, or of additional instruments, such as laser indirectophthalmoscopes, based upon a BIO platform. The apparatus, in preferredembodiments, will incorporate a material and electrical design so as tonot warp with normal use, or disallow heat dissipation by the BIOinstrument's illumination source, nor damage the manual BIO instrumentreversibly coupled to the device by the use of a custom-fit mountingbracket that can be, in a preferred embodiment, securely attached anddetached from the BIO instrument. In aspects, the internal opticalsystem surfaces will be matte black to reduce internal reflections/lightscatter and glare (e.g., Purkinje reflections/reflexes from the cornea,tear film, patient's natural lens, and handheld condensing lens, orother stray reflections from the device and BIO instrument's internaloptical system and optical/illumination transmission windows). Thedevice, in a preferred embodiment, will allow access to the BIOinstrument optical & illumination viewport and the device's own optical& illumination transmission window by simple detachment of the devicefrom the BIO instrument for cleaning these surfaces, so as to removedust, fingerprints, or other debris which may degrade opticalperformance. In another embodiment, non-corrosive coupling agents suchas films or gels applied between the BIO instrument viewport and thedevice transmission window may or may not be used to enhance opticalperformance between the device herein taught and the BIO instrumentadapted.

In some aspects, linear polarizers may be used to polarizelight-emitting diode (LED) light and images to the camera. Softwaretechniques and applications may also be used (e.g., frame doubling, highdynamic range [HDR] image processing algorithms), along with softwareimage stabilization techniques to optimize light and image qualitycaptured by the image sensor of imagery from the eye. These techniqueswould help optimize image quality and fidelity in settings wherehigh-quality image capture is challenging, given the operational need tocapture images with a low-enough shutter speed to capture a sharp imagedespite the natural movements and saccades of the patient eye beingexamined, the relatively dark background illumination of the examinationroom, the bright illumination source and numerous sources of glareduring the examination, and relatively darkly-pigmented ocularstructures in many patients. In certain examples often found in clinicalpractice, these techniques would help capture high-quality, well-exposedimagery in circumstances in which there is a large difference betweendark and bright areas of the image captured, in patients with dark fundi(relatively darkly-pigmented retina and other posterior-segmentstructures of the eye), or in patients who are quite light-sensitive andso the illumination levels used by the BIO instrument are relativelylow. Software techniques for glare reduction include, but are notlimited to, eliminating imagery captured at maximum image brightness, orclose to a 255 value of image brightness, when using grayscale pixelvalues in software image processing. To optimize image stabilization,hardware or software techniques may be used, such as automatic imagestabilization computer software hosted on the device or off-device, oroptical techniques such as isolation of the optical system to vibrationby the use of additional elements such as, but not limited to,vibration-dampening elements surrounding the optical components such asthe camera lens, image sensor, and plate beamsplitter, pivoted supportelements such as gimbal stabilizers with or without onboard motors, orthe use of internal springs mounted between the device externalenclosure and optical platform.

Triggering the camera to capture an image may be accomplished by way ofa manual or automatic handswitch, footswitch, or other networkedcomputing device, which may be wired or wireless. In some embodiments,other aspects of the apparatus may be placed in and/or around thefootswitch, such as a CPU, battery, processor, and/or other components.In a preferred embodiment, no user intervention would be required beyondthe conventional techniques of indirect ophthalmoscopy; for example,auto-capture of images would occur once an in-focus retina or part(s) ofthe retina are detected by a combination of integrated device elementsincluding, but not limited to, the image processor, integratedalgorithmic image processing software, additional sensors such as, inone embodiment, a distometer, and device firmware. This may becomputationally or algorithmically implemented. In one aspect, computersoftware detects whether a retina or portion of the retina is in view(e.g., based on color and/or other characteristics) and withinprogrammed tolerance limits for distance and focus, and begins and stopscapturing images automatically, either on-the-fly as high-quality fundusimagery is detected by the system by criteria such as discussed prior,or after a pre-programmed, adjustable number of images or time interval.For example, the software may stop capturing images and alert the userwhen an adequate capturing of the retina or portion of the retina isfinished. Other criteria for starting and stopping may be used. Forexample, using edge detection or other image processing techniques, thesoftware may only instruct image capture when an in-focus image isdetected. Alternatively, images may be captured using a remote triggervia a computer software application (“an app”) either via mobile device,web-based, or other means. By way of example, a person other than theexaminer may trigger image capture on a phone, tablet, or computer, andthe person may be physically located at the examiner site or in a remotelocation.

Upon capturing images, image export from the device onboard memory andfile storage system may occur automatically, or images may be manuallyretrieved and exported to a separate file storage system either integralto the device, or to a removable file storage apparatus (such as, forexample, SD card or a USB device). Images may also be exportedwirelessly. An automatic, user-accessible file system hierarchy may, inone embodiment, attach metadata to the captured imagery and organize theimages with little or no direct user intervention from the indirectophthalmoscopy examination session. In one aspect, these images may beencrypted to protect the health information of the patient to meetHIPAA/HiTech Act requirements. For a “store and forward” image captureand review system, systems for image capture, retrieval, review,redisplay, annotation, comparison, enhancement, and storage may includedirectly or via a compatible software integration with, in one aspect, asoftware image management, electronic health record or electronicmedical record (EHR/EMR) system, or document management system. Imagesmay, in aspects, use wireless communication protocols such as Bluetoothto send to a device, such as a mobile device, or WiFi Directtransmission to individual team members or pre-authenticatedhierarchical groups of authorized users (respectively here also referredto as “trusted users,” or “trusted groups”).

In a preferred embodiment, wireless image/data transfer is encrypted tomaintain information security and/or transmitted to trusted users only.Encryption of encoded and transmitted data may be performed on-device oroff-device, and in a preferred embodiment, network security will bemaintained across all associated networked access points and connecteddevices. In aspects, this wireless data transfer will occur via localretransmission using accepted wireless data transmission standards (suchas, but not limited to, WiFi and Bluetooth connections) to a mobiledevice, web app, or computer, such as between teacher(s) and student(s).In a preferred embodiment, radio frequencies and transmission standardsused for wireless communication between connected devices and thetransmitting device antenna array will balance goals of low-powerconnections, ease and stability of connectivity, data transmissionspeeds, line-of-sight requirements, distance of data transmissionrequired, number of wireless connections required per device, andaccepted standards and preferred radio frequencies for medical wirelessdata transmission to minimize electromagnetic interference with othertechnologies used in the medical environment. In a preferred embodiment,one-time auto-pairing between trusted user and device will happen;otherwise, sharing/pairing is set once each time the user is changed.

Regarding the microprocessor aspect of the apparatus, such as a computerprocessing unit (“CPU”), in one embodiment, the system used by thecamera and apparatus could be a microprocessor. Another option is to usea pre-processing chip on the camera, which are, for example, used incertain webcams, which might conduct the majority of the processingeither on a mobile phone, tablet, or computer, or in the handswitch orfootswitch. In a preferred embodiment, a system on chip (SoC) or systemon module (SOM) design platform will be used; advantages include but arenot limited to reduced size, reduced cost, and easier and enhancedmaintenance capabilities of the platform. In preferred embodiments,small, lightweight processors will be used to enable processing, and inone aspect, an all-in-one single board computer system could be used. Across-compiling toolchain may also be used. Integrated softwareapplications hosted on remote networked computing devices may also actas remote controllers of the device, such as, but not limited to, aweb-based app or an app on a mobile device or computer.

The use of onboard microprocessor, integrated image processingcomputational elements, wireless networking capabilities, and integrateddevice firmware and supported software applications together enableseveral other advantages beyond the prior art. Examples of theseadvantages include, but are not limited to, on-the-fly switching betweenone or more wirelessly-connected remote displays concurrent, orsubsequent to, the clinical examination session, without interruption ofthe image capture, device operation during indirect ophthalmoscopy, orvideo stream generated by the device and user in a particularexamination session. Additionally, integrating authentication ofclinical users with a particular device, location, and examinationsession, may enable clinical collaboration significantly beyond theprior art. In one embodiment, the use of authenticated device users,wireless image and video capture and transmission, wirelessbidirectional data transfer, onboard programmable microprocessor, remoteredisplay of imagery generated in a clinical examination session, andcompatible integrated off-device software applications can enableconcurrent or time-delayed communications between clinical colleagueseither in close or remote physical proximity via a separate clinicalsoftware application. Bidirectional wireless data transfer can enable,in one aspect, remotely-triggered image capture, remote switching ofdevice modes, or remotely enabling other device functions without theuser's need for direct intervention. Examples of these advantagesinclude, but are not limited to, the eye physician user checking imagequality of the capture session after capturing imagery from one eye ontheir connected smartphone; annotating the imagery with certain notesand flagging the image for review by a senior physician and sending aquiz question based upon the imagery to a junior colleague via thesoftware application user interface; a junior resident physiciansimultaneously viewing the examination session imagery via their ownconnected mobile device to learn the techniques of indirectophthalmoscopy and for vitreoretinal medical education purposes; one ormore remotely-located physicians observing the imaging sessionconcurrently with the examination or in a time-delayed fashion, makingclinical observations into the communications interface in compatibleInternet software applications during a Grand Rounds medical educationsconference; and a senior physician verifying the quality andcompleteness of the indirect ophthalmoscopy examination, switching imagecapture modes at a certain point from a nearby networked computingdevice in order to better capture a desired ocular feature whilesignaling the mode shift to the user using audio and haptic cues, allwithout an interruption in the medical examination process.

In one preferred embodiment, the embedded microprocessor and wirelessantenna array, along with integrated, secure remote networking softwaresuch as virtual private network (VPN) software, along with algorithmictechniques such as packet forwarding, will allow pre-authenticated,authorized (“trusted”) system technicians to troubleshoot, maintain, andupdate the device or groups of devices and integrated system remotelyand provide automatic periodic updates to enhance system stability,security, and enable automatic rollout of new software-enabled functionsand enhanced functionality of the BIO digital adapter system over time.This provides additional substantial advantages to the prior art, assubstantial improvements in device functionality and reliability can bemade remotely and wirelessly over time to one or more devices ordesignated groups of devices via software or firmware updates to thereprogrammable device microprocessor, and in one aspect, device errormessages or the need for system maintenance can be wirelessly sent tocompatible networked devices or off-device networked softwareapplication interfaces to signal system administrators that maintenancefunctions are necessary for a particular device (which, vialocation-based tracking as described prior, can help the systemadministrator pinpoint a specific device location and particular doctorswho may need replacement devices). In a particularly largemulti-specialty practice or hospital-based setting, these sorts ofremote device administration features enabled by the device taught hereoffer substantial advantages beyond tedious periodic inspection of eachdevice within a practice for already-busy practice personnel andancillary staff members.

The microprocessor and integrated on-device image processing, devicefirmware, and integrated off-device networked software applications, inone aspect, will allow for auto-montage of the retina and ocular fundus.Auto-montage can be described as the automatic, or software-assisted,joining, through one or more established or novel algorithmictechniques, of adjacent ocular fundus imagery captured throughindividual overlapping images or sets of images through the relativelynarrow field of view of the handheld condensing lens used in indirectophthalmoscopy, relative to the full span of the ocular fundus as viewedby the complete examination of each eye. The fundus montage would offera full “map” of the fundus of each eye in a single high-resolutionimage; have, in one aspect, the ability to compensate for distorted orincomplete capture of features in one or more individual image captures;provide an easily-magnified image that could be stored, retrieved, andcompared to prior examination montage images; and provide a “global”view of the fundus as a whole, as opposed to the narrow “keyhole” viewof individual images viewed under high magnification through thehandheld lens. The montage would typically be examined on an off-devicedisplay by the user or third-party reviewer at the end of theexamination, or at the beginning of a follow-up examination to, forexample, compare patient's examinations for progression versus stabilityof any clinical pathology found. In one aspect, a process would “stitchtogether” more than one image of the retina or ocular fundus (hereshortened to simply “the fundus”), or of other ocular features, toprovide a more complete picture of a larger portion of the retina orocular fundus than just one picture of one portion of the retina orfundus, allowing for quick review of the entire fundus at one time,rather than scanning through a library of images of successive portionsof the fundus. Several existing technologies, essentially imagestitching algorithms, may assist with the auto-montage feature, inaddition to or beyond any novel techniques used. These existingtechnologies will be familiar to one of ordinary skill in the art.On-device or post-processing enhancements may be assisted by the use ofembedded sensors on or enclosed in the device and connected peripheralelements such as accelerometers, infrared-based distance sensors, orgyroscopes, to automatically detect or suggest image orientation,portion of the fundus examined in an image frame, etc.

A sensor array may be provided on the imaging device disclosed hereinincluding one or more of the sensors as described directly above orelsewhere in the application.

The processor or other software may also allow for auto-crop andauto-center with a black frame typical of standard fundus photographs.Auto-crop may be described as the automatic, algorithmic removal ofextraneous imagery captured by the onboard system beyond the ocularfeatures desired to be captured, which may or may not be used incombination with the algorithmic placement of a standardized black framewith or without standard orientation marks as typically seen inhigh-quality fundus photographs. Auto-center may be described as theautomatic, algorithmic recognition of the circular handheld condensinglens ring held by the examiner and centration of this ring in thecaptured fundus photograph, with or without additional image processingtechniques such as adjustment for tilt of the lens (forming an ellipse,rather than a circle in the captured photograph) to minimize distortionsin the captured photograph where applicable. The images produced bythese techniques allow for easy comparison of fundus photos generated bythe system described herein to traditional fundus photographs thatexaminers typically inspect as generated by currently existing fundusphotography technology. The onboard device image processor or otherpost-processing software may also reduce/eliminate ambient lightingartifacts (such as, by way of an example, fluorescent lighting). Theprocessing or other post-processing software may include providing ared-free image or reduced-red image to better distinguish between blood,vessels, and the ordinarily orange or reddish features of the ocularfundus. The use of a reprogrammable, wirelessly networked devicemicroprocessor and integrated software applications hosted on- oroff-device allows a variety of options to exist to correct for imagerycaptured in a flipped or reversed orientation beyond the prior art,which largely must rely on optical elements to correctly orient theimagery captured, or subsequent manual manipulation of captured imageson separate computer software requiring substantial user intervention.In a preferred embodiment, correction of image or video orientation maybe performed in real time or near real time to the clinical examinationby on video processing of the camera output, which represents a standardimprovement versus direct mirror of video output. Furthermore,ophthalmic imagery captured may be in real time or near real timere-oriented by such software- and hardware-based techniques in thetaught system to their correct anatomic orientation (which is ordinarilyreversed and flipped in the aerial image as viewed through the handheldcondensing lens used in indirect ophthalmoscopy).

A variety of data science, image processing, and computer sciencealgorithmic techniques may be employed to further enhance the diagnosticand medical record-keeping capabilities of the integrated system heretaught. Algorithmic techniques such as, but not limited to, machinelearning-based automated image element recognition for the system mayalso be included as part of the device and system. Such technology maybe used to, for example, recognize that a focused retina is in view (toinitiate capture), to recognize which eye is examined, and when a higherlevel of image magnification is used or needed (for example, to capturea high-quality image of the optic nerve head of each eye), to locatelarge library/libraries of tagged fundus images (e.g., right or left)for algorithmic computer vision-based fundus photography image gradingusing computing applications and algorithms such as, but not limited to,TensorFlow, and/or rapidly collect large datasets of clinical imageryalone or in combination with clinical metadata for artificialintelligence-based healthcare automation software systems such as, toname one example, DeepMind Health. In one embodiment, optimized softwarelibraries for machine learning can be integrated with the devicemicroprocessor or image coprocessor to enable rapid acquisition ofalgorithmically enhanced or recognized imagery, balancing the power anddata storage limitations of the portable embedded system taught herein.Bidirectional data transfer and device control capabilities of theembedded system taught herein can also, in one embodiment, userecognized user and patient states based upon algorithmically-recognizedocular features to enable or simplify automatic or manual switchingbetween disparate imaging modes optimized for high-quality capture ofvarious desired structures imaged in and around the eye of the patientunder examination.

The battery and integrated device power management system, in apreferred embodiment, will feature a distinction between low-power andhigh-power states (also here referred to “sleep/wake” states), to switchfrom sleep to wake states when the user needs to use the apparatus. Forexample, in one embodiment, the device will “wake” whenever a user picksup the BIO instrument from the wall by use of onboard sensors such as anaccelerometer, thereby not requiring manually powering on the device bythe use of a physical switch each time a user puts a BIO on his/herhead. In some embodiments, the device may be charged by an AC adapter.In some embodiments the device will include a battery to support fullyuntethered use of the device. The battery may be an integratedrechargeable battery, or also could, in one embodiment, support a “hotswappable” array of user-replaceable batteries to extend use time overan extended clinical session; in one aspect, the battery may berecharged by a universal serial bus (USB) charging cable and compatiblealternating current (AC) adapter or separate power storage system. Inanother aspect, the device will be charged when placed into a dedicatedcharging station or holding base. In another aspect, the device may becharged from the BIO charging cradle either directly, by the use of acoupled charging adapter custom-fit to each model of charging cradle,such as those commonly used to charge “wireless” BIO instrument models(commonly described as “wireless,” in that they use onboard rechargeablebatteries, versus a wired AC power connection). In another embodiment,charging of the device could be conducted using an adjacent wired orwireless charging station (using, in one embodiment, wireless batterycharging technologies such as wireless induction coil-based charging),enabling the quick use and replacement of the device throughout a busyclinical workflow and the simple continued recharging of the device inbetween patient examination sessions.

Data formats for device capture, manipulation, storage, retrieval, andtransmission of data that is created and stored by the device arereferred to as, in some aspects, documents. A document may containmultiple blocks of data received from the hardware device. These blocksof data are referred to as pages. A document must contain at least 1page, but has no upper limit on the number of pages. An exception tothis is if there are errors on the device. In that case, a document withno pages can be returned, but the error collection will be filled in.Also, if there are errors, the document may still contain pages.However, these pages should be assumed to represent invalid data.Documents are grouped together in a session, which generally representsa patient exam. Sessions may contain documents obtained from multipledifferent hardware devices. Each session, document, and page within thedocuments may have searchable metadata that is not patient identifiable.This is to provide a quick means of searching the data captured andstored without having to decrypt every record.

In one aspect, the basic structure may appear as follows. A session maycomprise, but is not limited to: globally unique ID; MRN, which inpreferred embodiments is either an encrypted value, or a hash of a valuethat is stored elsewhere; Name; Description; Start Timestamp; EndTimestamp; Responsible User IDs; Documents; and Device Groups associatedwith the Session.

A Document may comprise, but is not limited to: globally unique ID;Device ID; Operator ID; Pages; Metadata entries; and Messages. Documentswill contain at least one Page or Message.

A Page may comprise, but is not limited to: globally unique ID; Data.Type Description; Data Format; Filename; DataURI (Data Uniform ResourceIdentifier), in a preferred embodiment the raw data is stored in adifferent data source to keep actual patient data isolated fromidentifying data; Timestamp; Metadata entries.

A Metadata entry may comprise, but not limited to: globally unique ID;Key; and Value.

A Message may comprise, but is not limited to: globally unique ID;Device ID; Device Message ID; Message Severity; Message Type; Text; andTimestamp.

A Device Group may comprise, but is not limited to: globally unique ID;Name; Description; Session ID; and Devices.

A Device may comprise, but is not limited to: globally unique ID; Name;Description; DeviceType; and Device Group ID.

A DeviceType may comprise, but is not limited to: ID and Name.

In other embodiments, different file formats may encode other types ofmetadata. Different devices or device versions may encode, in certainembodiments, the state of the software configuration, the state of thedevice, and calibration data, that excludes patient-specific orprotected health information. In certain embodiments, these may includean IP address of the device, the software and hardware version of thedevice, the device geolocation of the device, or other device- oruser-specific data useful to device management and interpretation of thedata but not specific to the patient-related data itself.

In embodiments, the above data format may be used for:

Sessions

Sessions, in embodiments, may be analogous to patient visits. There willtypically be a one-to-one correspondence between a session and a patientvisit. In aspects, sessions are not connection instances since a webservice may be used; accordingly, connections may not necessarily beheld open. Sessions can be created, started, stopped, etc. via awebsite, for example. This allows the workflow of associating devicedocuments with a session that is not associated with an MRN, and thenallowing a user to manually split that data into sessions linked toMRNs. For example, a user could use the timestamps of the documents.

Documents

Documents may comprise, in aspects, a single “upload” of data from aDevice to a Session. In aspects, a Document will be one Page; forexample, a fundus image. In other aspects, a Document will comprisemultiple pages, allowing for, for example, a burst of images from thedevice and uploading the images in one step, saving different fileformats, saving other types of data along with the data (e.g., audio,left and right eye images, etc.). In one embodiment, a Document may besent from a device to a Session that contains only a Message. Messages,in one aspect, comprise a data type well-suited for devicestatus-related information such as, for example, “Battery Level,” asthey include, in aspects, Acuity and Message Type. In this example, ifthe device onboard battery is at 20% charge, the user may receive aWarning Message, but at 5% charge, the user could, in this embodiment,receive a Critical Message. In this embodiment, as the device detects alow battery charge, the system may send a device status-related messagewithout being restricted by whether the user captures an image.

Pages

Pages, in aspects, comprise raw data from the Device, along with dataformat info for storage information.

Metadata

Metadata, in aspects, is used to store “out of band” data that may beuseful to the user. This may include, but is not limited to, calibrationdata, software/hardware versions, geolocation data, etc.

Devices

Device records, in aspects, may comprise data for the user and theserver to identify a device that is registered with the system.

Device Groups

Device Groups, in aspects, comprise collections of Devices that can beassociated with a Session. For example, this may be used to set up allDevices in a particular examination room to be grouped in a DeviceGroup. When a user creates a Session for a patient, the information maykey, for example: “Patient will be in Examination Room B; associateExamination Room B Device Group with the Session.” Accordingly, alldevices in Examination Room B are automatically linked to the Sessionand will send accompanying Documents to the correct data location.Device Groups, in aspects, may contain a single device if it is desiredthat devices not be grouped together.

The system taught herein may also be used as part of or in electronic(wired or wireless) bidirectional connection with a paired datamanagement control device (here referred to as a “hub”) to manage andorganize devices, examinations, and people involved in the examinationprocess and associated data produced by each correctly and across theclinical data network. The hub comprises a processor (e.g., a CPU) andmay be connected to the Internet with wire(s) or wirelessly. It may alsobe unconnected from the Internet in a dedicated local area network (LAN)or wide area network (WAN). In a preferred embodiment, the hub will bewirelessly connected to devices or examiners in the examining facilityto monitor activity and permit multiple device control and coordination.In one aspect, the hub will receive images and data/information from thedevice taught herein or other devices. It will be used, along withuniquely identifiable markers such as hardware tokens, paired mobiledevices, or passcodes, to detect and manage the hierarchy of trustedusers engaged in use of a connected network of devices as previouslydescribed. It will break down the data, review the data, analyze thedata, manage for storage, sync images and information, process images orinformation, and/or manage remote data synchronization and/or local orremote redisplay. It may also manage storing such information locally orremotely.

For example, the hub may be connected to several devices taught hereinwithin a facility. The hub will record when such devices are being usedand who is using the devices to automatically, or with minimal userintervention, maintain a secure hierarchical audit trail of clinicaldata generated by the clinical data network here described. The hub willlog, save, organize, and process such information in order to, amongother things, know when examinations were being performed, what kind ofexaminations were being performed, and who was performing suchexaminations. Personnel in or around the facility may be tracked, in oneaspect, by having a radio-frequency identification (RFID)- or near fieldcommunications (NFC)-compatible tracking device on their person, or bytracking a paired mobile phone or some other electronic devicewirelessly by a variety of hardware-assisted and algorithmic softwaremethods such as, but not limited to, location-based and time offlight-based wireless tracking. The information collected by the hub maybe cross-referenced with other information, for example a referenceclinical schedule, to track activity in or around the facility. Inanother example, the hub may automatically, or with user input, pairpatient imagery and metadata collected during an examination with thespecific patient and associated doctor at that appointment time, withthe hub subsequently associating and exporting collected data to thepatient's EMR/EHR based on the reference clinical schedule used.

Now turning to the Figures, FIG. 1 shows a computer-generated image ofthe device as an assembled enclosure apparatus (e.g., optics, camera,and electronic elements (not shown, for clarity)), including the BIO1100, reversible mounting bracket 1000, and optical cartridge 1200,which optionally fits into or on the mounting bracket). FIG. 2 shows athree-quarters exploded view of the optical cartridge 2200, mountingbracket 2000, and sample BIO 2100 of FIG. 1 split into three separatecomponents, with dashed lines indicating alignment of the respectiveelements for assembly, in this particular embodiment. Regarding the BIOand its viewport, this component may vary depending on the make/model ofthe BIO instrument and the device taught herein will allow for differentmounting brackets to fit the varying types of BIOs and/or BIOviewpieces/viewports, by manufacturer. Optionally, the bracket may beadjusted to fit different viewpieces with, for example, sliding pieces,snap-in-place mounting elements, adjustable knobs, or other means ofadjusting the bracket so it can be adjusted to optimally fit differenttypes of BIO view ports/BIO viewpieces/BIOs without requiringdisassembly, modification, or damage to the BIO instrument beingadapted.

The cartridge 1200, 2200 is optional. The optics and other componentsmay be on the mounting bracket or otherwise incorporated into or ontothe mounting bracket and not necessarily incorporated by a removablecartridge. In one embodiment, the cartridge is incorporated as part of asingle unit including the mounting bracket and all optical andelectrical components, which in an embodiment, is a single molded piece(or interlocking piece) incorporating the elements of the imaging devicetaught herein. The optional cartridge, if the embodiment includes acartridge, may be removably attached to the mounting bracket or BIO 1100using one or more of the following:

-   -   a. a bracket mechanism,    -   b. a sliding mechanism,    -   c. a clip mechanism,    -   d. an expansion joint mechanism,    -   e. tension spring(s),    -   f. magnets(s),    -   g. fastening tab(s),    -   h. interlocking component assembly,    -   i. molded-fit assembly,    -   j. press-on assembly,    -   k. hook-and-eyelet,    -   l. snap fastener(s),    -   m. snap-fit joint(s),    -   n. removable adhesive,    -   o. interlocking joint(s),    -   p. snap-on joining mechanism,    -   q. cantilever joining mechanism,    -   r. U-shaped, torsional, and/or annular snap-fit joining assembly        techniques, and/or    -   s. mechanisms in which a protruding part of one component is        deflected during a joining operation and catches in a depression        in a mating component.

Full or partial adjustment of BIO controls is permitted by the device'smounting bracket and device enclosure design elements (such as, forexample, levers/knobs (e.g., 1500 in FIG. 1) through specially-designedmounting bracket cutouts, and other means for adjusting the device tofit different BIOs and BIO view pieces/viewports).

The optical cartridge 1200, in the pictured embodiment, fits into or onthe mounting bracket 1000. A beamsplitter may fit into a slot 1300 inthe optical cartridge. As pictured, for example, a beamsplitter glassfits into the pictured slot at a 45-degree angle to the incident ray andthe beamsplitter generally fits into that area of the cartridge. In oneaspect, above the area for the beam splitter, a fitting or slot for thecamera is situated 1400, in which a camera (e.g., optics, lens, imagecapturing means, electronics, and focusing/adjusting mechanism) may beplaced and secured into desired optical alignment with the BIO viewport,its internal optical and illumination system, and the view through theBIO instrument eyepieces. The view is centrally coaxial and identicalwith the examiner's view. On the top of the cartridge, as pictured, anarea exists for additional components, although components may belocated elsewhere on the device or not on the device. Such componentsinclude, but are not limited to, a battery charger, an accelerometer, awireless or wired charging means or connection, a WiFi or Bluetooth chip(wireless network chip), a battery, a microprocessor, an antenna(s), amicrophone, a speaker, a power controller, switches (for example, foron/off or other commands), and status LED(s) (for example, forindicating on/off, error, ready for certain commands, etc.). The statusLEDs will be designed, in a preferred embodiment, to be viewed at adistance, while remaining unobtrusive during the eye examination in adarkened examination room, while the BIO instrument and attached deviceis positioned in its wall-mounted battery charging cradle.

In one embodiment, a microprocessor, antenna(s), and power supply ispositioned as a unit into the top of the optional optical cartridge, thecamera assembly is positioned as a unit into the corresponding cavity inthe middle section below the support risers, and the beamsplitter andoptical and illumination transmission window or cutouts for BIOillumination source and viewport are in the bottom section. In oneembodiment, the pictured apparatus slides into place as one section likea cartridge into a mounting bracket adapter and, in an embodiment, therewill exist different modular mounting brackets to fit different BIOinstrument models.

FIG. 2 shows another embodiment of the device. The mounting bracket 2000with optional cartridge 2200 is shown (along with the area or slot for acamera 2400). The BIO is shown as 2100 with BIO adjustment levers orknobs 2500. In this depiction, the BIO front viewport or front viewingwindow is shown 2600. Also, the BIO illumination window is shown 2700.Finally, the mounting bracket transmission window is depicted 2800.

FIG. 3 shows a flowchart of the high-level architecture of the systemsoftware application(s) used to implement the intended purpose of theimaging device taught herein. Specifically, a block diagram is showndepicting data transmission and conceptual hierarchy or software layersand services. In one embodiment, at the command receiver level, theimage captured by the device(s) 3000 will be sent by Bluetooth, WiFi, bywire, or by OnDevice listeners. The data may be encrypted 3001. Next,the command service function is performed, wherein a command bus handlesinfo command and session command 3002. Clinical data capture sessionsare then managed and if there is a current session, information flows tothe session repository proxy 3003 and then to the data storage layer3004 and information such as documents, information, images, and/ormetadata are stored in a session repository before moving to maintenance3005. At the maintenance level, the documents, information, images,and/or metadata are backed up to cloud-based storage 3006.

FIG. 4 is a three-quarters view close up of another embodiment of thecentral optional optical cartridge 4200 described herein, without opticsand electronic elements. In the bottom opening, the slot for the beamsplitter glass is shown 4300. In one aspect, 30% of the light (image)gets bounced 90° up to the miniaturized camera, and 70% passes straighton to the viewport of the BIO unobstructed. The camera location or slotfor the on-board camera is shown at 4400.

FIG. 5 shows an exploded view of major components of assembly of theapparatus taught herein, including depictions of the apparatus assembledand disassembled into separate parts. As depicted, an illuminationtransmission window 5000 allowing passage of light from the BIOillumination source is shown along with an optical transmission window5001 through which incident light from the eye passes to the BIOviewport and to the examiner. On the other side, a viewing window 5002allows for transmission of light to the BIO viewport and then to theexaminer. A computer module housing is shown at 5003 to which, forexample, a single or multiple board computer, or system on module, 5004is associated. The one or more camera is shown in 5005. And an opticalhousing 5006 for the camera and beamsplitter 5007 is shown which alignsthe camera elements above and coaxial with the reflected image of theeye under examination. The housing also provides space for theconverging optical elements, such as the incident light, on-boardcamera, beamsplitter, and outgoing illumination beam, and is coated witha matte black finish, in a preferred embodiment, to reduce glare fromstray or unwanted reflected light sources.

FIG. 6. shows a front-facing view of the BIO augmented imaging adapterassembly, including a BIO 6100, a BIO viewport 6600 with optionalpolarization (e.g., linear polarization) of light to minimize glare, aBIO illumination window 6700 with optional polarization (e.g., linearpolarization) of light to minimize glare, a front-mounted mountingbracket 6000, optical assembly (optionally an attached and/or removablecartridge) 6200, camera 6400, microprocessor 6900, andantenna/battery/sensor array configuration 6901. The existing BIOmounting adjustment knobs 6500 are fully accessible due to custommounting bracket design with device-specific cutouts for adjustmentmechanisms, even with the inventive device explained herein attached andfunctional. The microprocessor, wireless antenna, battery, sensor arrayhoused in the optional optical cartridge allow for image/videotransmission and/or processing, system control via multiple mechanisms(mechanical, voice, mobile application, local or remote computercontrol, handheld or footswitch-type remote controller), andbidirectional data transmission and telemetry.

The device allows for full or almost full transmission of the existingBIO illumination source, but may also include additional sources ofillumination, and an optional polarization of light to minimize glare.

FIG. 7 is a side-view of the BIO augmented imaging adapter assembly,including the BIO 7100, front-mounted bracket assembly 7000, opticalassembly 7200, camera, and microprocessor/antenna/battery/sensor array.Knobs and levers 7500 on the existing BIO device are unobstructed by themounting adapter design in preferred embodiments, allowing for fullfunctionality of the BIO device as familiar by examiners. The mountingbracket allows for flush mounting against the BIO viewpiece for preciseimage alignment of the camera with the examiner's view. The BIO viewingeyepiece 7101 as adapted to by the device taught herein allows forunobstructed, direct view from multiple models of BIO instruments to theembedded camera system. The mounting bracket may be adjustable toaccount for different makes/models of BIO instruments, or it may becustom-designed to fit particular makes/models. The BIO viewing eyepiecealong with the imaging device design allows for unobstructed, directview by an examiner and coaxial redisplay of the view from multiplemodels of BIO instruments to the embedded camera system.

In FIG. 8. is a top-down view of the BIO augmented imaging adapterassembly, including the BIO 8100, front-mounted bracket 8000, andoptionally removable optical cartridge 8200. In embodiments, thelow-profile, interchangeable mounting bracket assembly allows forstable, precise optical alignment and capture of examination imagery andsensor data, without impeding the view, interfering with clinicalexamination, or impeding ergonomics. The enclosure on the opticalcartridge for the microprocessor and other components, such as thecamera, and microprocessor/antenna/battery/sensor array, is prominentlyfeatured 8201. In preferred embodiments, towards the top of the opticalcartridge is where the components will be enclosed. In a preferredembodiment, the BIO instrument eyepieces 8101 have an interpupillarydistance adjustment mechanism(s), which remains fully adjustable basedon the device-specific design of the imaging device. The BIO instrumentillumination height adjustment knobs and instrument illuminationadjustment levers 8500 remain unobstructed and accessible.

FIG. 9 shows a front-facing view of the BIO augmented imaging adapterassembly, including a BIO 9100, BIO viewport 9600, front-mountedmounting bracket 9000, and optical cartridge 9200. The mounting bracket,as explained, may be coupled or decoupled from the BIO viewport, and theoptional optical cartridge may be removed from the mounting bracket, orit may be attached, integrated, or molded as one with the mountingbracket. In this depiction, the optional optical cartridge has beenremoved from the mounting bracket and separated; in embodiments, thecartridge will be optionally removable (either for ease of manufacturingand assembly, or by the end user), such as by sliding out of themounting bracket. This removability allows for maintenance, repair andreplacement, ease of access to replacement parts, updates, charging,switching in and out components, such as the camera, microprocessor,antenna, battery, and or sensor array, and/or repairing or otherwisemanipulating components.

FIG. 10 is a schematic diagram of the optical cartridge componentassembly focusing on the computational, power, and data storageelements. The optical transmission window 1001 houses the beamsplitter.The illumination source window 1002 allows for transmission of lightfrom the BIO illumination source. The camera and optical elements 1003are contained in the optional cartridge as depicted. The componentassembly also includes a power regulator and/or charger, 1004 as well asa battery or other power source 1005. It may include a charging port1006 and possibly a removable storage device port 1007. The cartridgemay include a microprocessor 1008, sensor array 1009, and/or antenna1009.

FIG. 11 is three-quarters view of the BIO augmented imaging adapterassembly, including the BIO 1110, front-mounted bracket assembly 1100,beamsplitter 1109, one or more camera 1101, camera lens 1102, and focusadjustment lever 1103, image sensor(s) and image coprocessor(s) 1104,the microprocessor and other electronic component(s) and/orcontroller(s) 1105, and the battery 1106. Adjustment knobs and levers1150 on the existing BIO device are unobstructed by the mounting adapterdesign in preferred embodiments, allowing for full functionality of thedevice as familiar by examiners.

This diagram further demonstrates the correspondence between the fieldof view and angle of view between the BIO viewport and optical system(what the examiner sees) and the imaging device optical system (what thecamera sees) as configured in the optical system of the optional opticalcartridge (not depicted). For example, 1107 shows the viewing angle andfield of view from the BIO viewport that shows the area that can be seenwhile examining the eye, which is what the examiner sees. 1108 shows theviewing angle and field of view of the camera system, which ispositioned and configured so as to correspond to the reflected image ofthe eye under examination as view through the BIO viewport.

FIG. 12 is side-facing, cutaway diagram of the BIO augmented imagingadapter assembly, including the BIO 1210, front-mounted bracket assembly1200, beamsplitter 1209, one or more camera 1201, camera lens 1202, andfocus adjustment lever 1203, image sensor(s) and image coprocessor(s)1204, the microprocessor and other electronic component(s) and/orcontroller(s) and sensor and network/antenna array 1205, and the battery1206. Adjustment knobs and levers 1250 on the existing BIO device areunobstructed by the mounting adapter design in preferred embodiments,allowing for full functionality of the device as familiar by examiners.

This diagram further demonstrates the correspondence between the fieldof view and angle of view between the BIO viewport and optical system(what the examiner sees) and the imaging device optical system (what thecamera sees) as configured in the optical system of the optional opticalcartridge (not depicted). For example, 1207 shows the viewing angle andfield of view from the BIO viewport that shows the area that can be seenwhile examining the eye, which is what the examiner sees. 1208 shows theviewing angle and field of view of the camera system, which ispositioned and configured so as to correspond to the reflected image ofthe eye under examination as view through the BIO viewport.

One skilled in the art will recognize that the disclosed features may beused singularly, in any combination, or omitted based on therequirements and specifications of a given application or design. Whenan embodiment refers to “comprising” certain features, it is to beunderstood that the embodiments can alternatively “consist of” or“consist essentially of” any one or more of the features. Otherembodiments of the invention will be apparent to those skilled in theart from consideration of the specification and practice of theinvention.

It is noted in particular that where a range of values is provided inthis specification, each value between the upper and lower limits ofthat range is also specifically disclosed. The upper and lower limits ofthese smaller ranges may independently be included or excluded in therange as well. The singular forms “a,” “an,” and “the” include pluralreferents unless the context clearly dictates otherwise. It is intendedthat the specification and examples be considered as exemplary in natureand that variations that do not depart from the essence of the inventionfall within the scope of the invention. Further, all of the referencescited in this disclosure are each individually incorporated by referenceherein in their entireties and as such are intended to provide anefficient way of supplementing the enabling disclosure of this inventionas well as provide background detailing the level of ordinary skill inthe art.

The invention claimed is:
 1. An imaging device for a binocular indirectophthalmoscope, comprising: one or more camera, wherein the one or morecamera is optionally provided by a cartridge comprising the one or morecamera; a mounting bracket capable of removably attaching the one ormore camera or the cartridge to a binocular indirect ophthalmoscope;wherein the mounting bracket comprises an opening or window configuredsuch that when the mounting bracket is attached to the one or morecamera or cartridge and to the binocular indirect ophthalmoscope, theopening or window permits incident light from an eye or portion of aneye being examined to transmit to a patient-facing front viewport of thebinocular indirect ophthalmoscope; one or more computer processing unit,memory unit, and/or communication device for sending one or moreelectronic still and/or video images from the one or more camera to anelectronic device by way of a wired or wireless connection and/or forstoring the one or more electronic still and/or video images on the oneor more memory unit; a beamsplitter, wherein the incident light from theeye or portion of the eye being examined is partially transmitted to thepatient-facing front viewport or a front viewing window of the binocularindirect ophthalmoscope and partially directed to the one or morecamera; and a power source.
 2. The imaging device of claim 1, whereinthe mounting bracket comprises an opening or window configured such thatwhen the mounting bracket is attached to the one or more camera orcartridge and to the binocular indirect ophthalmoscope, the opening orwindow permits transmission of light from an illumination source of thebinocular indirect ophthalmoscope.
 3. The imaging device of claim 1,wherein the one or more computer processing unit, memory unit, and/orcommunication device is capable of sending and/or receiving informationassociated with the one or more electronic still and/or video images. 4.The imaging device of claim 3, wherein the one or more communicationdevice enables bi-directional networked image or video capture,redisplay, control, and/or transmission of the one or more electronicstill and/or video images and/or associated data.
 5. The imaging deviceof claim 3, wherein the one or more communication device enablesbi-directional networked automation of: filing and/or organization ofthe one or more electronic still and/or video images; and/or associationof the one or more electronic still and/or video images with a specificpatient(s), an examining physician(s), date(s) of an examination(s),specific ocular feature(s) detected by the device, specific device(s)used, image quality data, operational data of the device(s), and/orlocation(s) of the device(s) or examination(s).
 6. The imaging device ofclaim 1, wherein the mounting bracket is removably attached to thebinocular indirect ophthalmoscope using one or more of the following: a.a bracket mechanism, b. a sliding mechanism, c. a clip mechanism, d. anexpansion joint mechanism, e. tension spring(s), f. magnets(s), g.fastening tab(s), h. interlocking component assembly, i. molded-fitassembly, j. press-on assembly, k. hook-and-eyelet, l. snap fastener(s),m. snap-fit joint(s), n. removable adhesive, o. interlocking joint(s),p. snap-on joining mechanism, q. cantilever joining mechanism, r.U-shaped, torsional, and/or annular snap-fit joining assemblytechniques, and/or s. mechanisms in which a protruding part of onecomponent is deflected during a joining operation and catches in adepression in a mating component.
 7. The imaging device of claim 1,wherein removably attaching the mounting bracket to the binocularindirect ophthalmoscope does not require disassembling or altering ofthe binocular indirect ophthalmoscope.
 8. The imaging device of claim 1,wherein the electronic device or a remote device allows for alteringsettings on and/or operation of the imaging device, controlling theimaging device, and/or triggering capture of the one or more electronicstill and/or video images.
 9. The imaging device of claim 1, wherein thebeamsplitter is a linear plate beamsplitter.
 10. The imaging device ofclaim 1, wherein the beamsplitter is capable of producing a singlyreflected image of an eye or structures within an eye under examination.11. The imaging device of claim 1, wherein light from an eye underexamination passes through the beamsplitter and to a single triangularmirror block associated with an optical system of the binocular indirectophthalmoscope.
 12. The imaging device of claim 1, further comprising asensor array.
 13. The imaging device of claim 1, further comprising afootswitch controller to trigger capturing the one or more electronicstill and/or video images using the one or more camera.
 14. The imagingdevice of claim 1, wherein the electronic device is chosen from a phone,computer, tablet computer, server, laptop computer, television, monitor,sensor, computer processing unit, and/or Internet-, local area network-,or wide area network-connected device.
 15. The imaging device of claim1, wherein the electronic device is used to perform one or more of thefollowing tasks: a. control settings on or related to the imagingdevice; b. to operate the imaging device; c. to redisplay the one ormore electronic still and/or video images captured from the imagingdevice; and/or d. to transmit, receive, analyze, gather, or collectinformation associated with the imaging device and/or the one or moreelectronic still and/or video images captured from the imaging device.16. The imaging device of claim 1, wherein the one or more still and/orvideo images are processed by locally stored computer softwareapplications, computer software applications stored remotely on aseparate networked device(s), or computer software applications storedon a remote server(s).
 17. The imaging device of claim 1, wherein theone or more camera provides an extended focal plane and/or field of viewand is capable of capturing the one or more still and/or video imagescorresponding to portions of a human or animal eye as viewed by a userof the binocular indirect ophthalmoscope.
 18. The imaging device ofclaim 1, wherein the mounting bracket is capable of positioning the oneor more camera or cartridge on a patient-facing side of the binocularindirect ophthalmoscope.
 19. The imaging device of claim 1, wherein theone or more camera is positioned above and between the eyepieces of thebinocular indirect ophthalmoscope, and wherein the one or more camera ismounted centrally or paracentrally to incident light beams in a visualaxis of the binocular indirect ophthalmoscope.
 20. The imaging device ofclaim 1, further comprising a lens or combination of lenses having anf-number greater than f−1.8.
 21. The imaging device of claim 1, whereinthe mounting bracket is aligned flush with a patient-facing viewport onthe binocular indirect ophthalmoscope.
 22. The imaging device of claim1, wherein the mounting bracket allows access to and operability ofadjustment knobs or levers on the binocular indirect ophthalmoscope. 23.The imaging device of claim 22, wherein the adjustment knobs or leverson the binocular indirect ophthalmoscope allow for adjusting one or moreof: a. interpupillary distance; b. transmission and/or manipulation ofthe illumination source of the binocular indirect ophthalmoscope; c.instrument illumination intensity, aperture, and/or angle of thebinocular indirect ophthalmoscope; d. optical alignment; e. stereopsis;f. viewing angle; and/or g. position of the binocular indirectophthalmoscope on a user's head.
 24. The imaging device of claim 1,wherein the one or more computer processing unit allows for optical-and/or sensor-assisted algorithmic techniques chosen from one or more ofthe following: a. focus stacking; b. inverting or reorienting the one ormore electronic still and/or video images; c. removing or reducing glareor visual occluding elements from the electronic still and/or videoimages; d. automatic capturing of the one or more electronic stilland/or video images when a retina, an optic nerve, and/or other portionof an eye is detected and in focus; e. comparing the one or moreelectronic still and/or video images captured by the one or more camerawith a library of images to detect which eye is being examined, detectan optic nerve, retinal vasculature, macula, or retinal periphery,and/or detect abnormal features of an eye being examined; f.software-selectable focusing planes; g. expanded depth of field imaging;h. region of interest-based focusing; i. exposure and focus controllingto ensure proper focus and exposure without or with minimal userintervention in routine clinical examination settings; j. constructionof a 3-dimensional view(s) of an eye and/or ocular structure; k.construction of a high dynamic range image(s) of an eye and/or ocularstructure; l. automatic montaging of overlapping adjacent electronicstill and/or video images for more complete representation of a retina,fundus, and/or other portion of an eye; and/or m. electronic annotationof observations.
 25. The imaging device of claim 1, further comprisingcomponents communicating visual and/or nonvisual ambient notificationsto a user of the binocular indirect ophthalmoscope chosen from one ormore of visual cues, audio cues, and/or haptic feedback.
 26. The imagingdevice of claim 1, further comprising: a first linear polarizer topolarize outgoing illumination from the illumination source of thebinocular indirect ophthalmoscope through the opening or window in themounting bracket; and a second linear polarizer to polarize incominglight to the one or more camera.
 27. A method of examining an eye of ahuman or animal, comprising: providing a fundus camera operablyconnected to one or more sensor and one or more processor; providing abinocular indirect ophthalmoscope removably attached to the funduscamera, wherein a user of the binocular indirect ophthalmoscope has theability to examine a patient's eye through the binocular indirectophthalmoscope while simultaneously capturing one or more still and/orvideo images of the patient's eye or portion of the patient's eye to beviewed in real-time or at a later time; electronically and automaticallycollecting information associated with an eye examination, the eye beingexamined, the patient being examined, and/or an examiner examining thepatient's eye; electronically and automatically integrating andcorrelating the information associated with the eye examination, the eyebeing examined, the patient being examined, and/or the examinerexamining the patient's eye and the one or more still and/or videoimages of the patient's eye or portion of the patient's eye; andelectronically transmitting the integrated and correlated informationassociated with the eye examination, the eye being examined, the patientbeing examined, and/or the examiner examining the patient's eye and theone or more still and/or video images of the patient's eye or portion ofthe patient's eye to an electronic device.
 28. The method of claim 27,further comprising using the integrated information associated with theeye examination, the eye being examined, the patient being examined,and/or the examiner examining the patient's eye and the one or morestill and/or video images of the patient's eye or portion of thepatient's eye to perform one or more of the following: diagnose thepatient; treat the patient; schedule other examinations or appointments;communicate with the patient, the examiner, or third parties; and/orcreate a database or library of all or part of the integratedinformation associated with the eye examination, the eye being examined,the patient being examined, and/or the examiner examining the patient'seye and the one or more still and/or video images of the patient's eyeor portion of the patient's eye.
 29. The method of claim 27, wherein theelectronic device is chosen from a phone, computer, tablet computer,server, laptop computer, television, monitor, sensor, computerprocessing unit, and/or Internet-, local area network-, or wide areanetwork-connected device.
 30. The method of claim 27, wherein the funduscamera is attached to a mounting bracket, the mounting bracket beingremovably attached to a binocular indirect ophthalmoscope.